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Abstract 

Many networks are characterized by highly heterogeneous distributions of links, 
which are called scale-free networks and the degree distributions follow p(k) ~ 
ck~ a . We study the robustness of scale- free networks to random failures from the 
character of their heterogeneity. Entropy of the degree distribution can be an average 
measure of a network's heterogeneity. Optimization of scale-free network robustness 
to random failures with average connectivity constant is equivalent to maximize the 
entropy of the degree distribution. By examining the relationship of the entropy 
of the degree distribution, scaling exponent and the minimal connectivity, we get 
the optimal design of scale- free network to random failures. We conclude that the 
entropy of the degree distribution is an effective measure of network's resilience to 
random failures. 
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1 Introduction 



Many complex systems are characterized by the network of interactions 
among its components. It has been shown that most of the networks are highly 
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heterogeneous in their connectivity patterns. Heterogeneity can be easily iden- 
tified by looking at the degree distribution p(k), which gives the probability 
of having a node with k links. Most complex networks can be described by 
degree distributions p(k) ~ ck~ a , with a G (2,3). These networks include so- 
cial networks(such as movie-actor networks, science citations and cooperation 
networks), the Internet and the World Wide Web, metabolic networks and 
protein interaction networks, etc [1,2,3,4,5]. 

Since Albert et al. raised the question of random failures and intentional 
attack on networks [6], enormous interest has been devoted to the study 
of the resilience of networks to failures of nodes or to intentional attacks 
[7,8,9,10,11,12]. Magoni studied a general strategy of attacks on the Inter- 
net [13]. It is important to understand how to design networks which are 
optimally robust against both failures and attacks. 

Most of researchers use percolation theory to study this problem [7,14]. 
A fraction p of the nodes and their connections are removed randomly— the 
integrity might be compromised: for a > 3, there exists a certain threshold 
value p c . When p > p c , the network disintegrates into smaller, disconnected 
parts. Below that critical threshold value, there still exists a connected cluster. 
For 2 < a < 3 , the networks are more resilient and the threshold value p c ap- 
proaches 1 [7] . Also, some researchers work on optimizing network robustness 
to both random failures and attacks with percolation theory [15,16,17]. 

A simple but essential character of scale-free network is its heterogeneous 
link distribution. Moreover, heterogeneity is in direct relationship with the 
network's resilience to attacks. Many real-world networks are scale-free and 
they are robust to random failures but vulnerable to targeted attacks. Het- 
erogeneity can be measured by entropy [18,19]. Sole et al. used entropy of the 
remaining degree and mutual information to study some model networks of 
different heterogeneity and randomness [19]. 

In this paper, different from percolation theory, we try an alternative point 
of view— the entropy of the degree distribution to describe scale-free network's 
heterogeneity. To determine the optimal design of scale-free network to ran- 
dom failures, we maximize the robustness of scale-free network to random 
failures while keeping the cost constant (that is, the average number of links 
per node remains constant). We find that optimization of scale-free network's 
robustness to random failures is equivalent to maximize the entropy of the 
degree distribution. By optimizing the entropy of the degree distribution, we 
get the scenario of optimal design of scale-free network to random failures. 
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2 Entropy of the degree distribution 



In general, scale-free networks have the degree distributions p(k), p(k) ~ 
ck~ a , k — m, m + 1, . . . , K, where m is the minimal connectivity and K is 
an effective connectivity cutoff present in finite networks. Then the entropy of 
the degree distribution can be defined as follows: 



N-l 

H = - p(k)logp(k) 
k=i 



where iV is the total number of nodes in the network and the restriction to the 
degree distribution is p(k) ~ ck~ a . Within the context of complex networks, 
the entropy of the degree distribution provides an average measure of network's 
heterogeneity, since it measures the diversity of the link distribution [19]. Two 
extreme cases are the maximal value and the minimal one. The maximum 
value is H max = log (N — 1) obtained for p(k) = jfzi(Vk = 1, 2, . . . , N — 1). 
H m in = occurs when p(k) = {0, . . . , 1, . . . , 0}. 

For the power-law degree distribution of scale-free network, the impact 
of diversity (long tails) is obvious, increasing the uncertainty. As the scaling 
exponent increases or the cut-off decreases, the network becomes less hetero- 
geneous and as a result a lower entropy is observed [19]. 

In this paper, we study the entropy of the degree distribution for scale-free 
networks. With continuous approximation, for the power law degree distribu- 
tion of scale-free network, entropy H can be expressed as follows: 



rK 

H = — p(k) log p(k)dk 

J m 

= - [ K ck- a log (ck~ a )dk 

Jm 
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1 ■ (K l - a - m 1 -")] (2) 
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The largest connectivity K pa mN"- 1 [7], The coefficient c is obtained from 
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p(k)dk 



ck~ a dk 



1 — a 



. — m 



l-cn 



(3) 



and thus 



(a — 1) ■ m 



a-l 



(4) 



Then entropy H can be expressed as a function of scaling exponent a, the 
minimal connectivity m and network scale N. 



H(a,m,N) = (log ( 



a — 1, 



in 



a 



1 — a' 



'iV 



a — 1 



logiV 
N 



(5) 



Fig. 1 displays the relationship between entropy if and a for different 
when the minimal connectivity m is defined. We know that with the increase 
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Fig. 1. H for different N versus a with m = 1. 

of scaling exponent a, the cut-off connectivity K decreases, which makes the 
network less heterogeneous. As a result, a lower entropy is observed. So differ- 
ent scaling exponent a brings the network different degree of heterogeneity. 

The average connectivity < k > is obtained with continuous approxima- 
tion, 



< k > 



K k-p(k)dk = j K k-rir 'ilk 



a 



■ (K 2 ~ a - m 



2-a\ 



(6) 
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and thus 



(X — 1 2 — a 

< k >~ m-(N~-l) (7) 

Zi ex 



According to the above results, we can formulate the scale-free network's 
resilience to random failures problem as the following nonlinear mixed integer 
programming. 



max H(a, m, N) 
s.t. < k >= const 

(8) 

The problem mentioned above is different from the usual maximizing the 
entropy of the network problem. Note that the highest entropy for a network 
with a given < k > is exponential random graph [20] . To get the optimal design 
of scale-free networks, we restrict the degree distribution to be a power-law 
distribution. The problem can be rewritten as follows: 



s.t. 



K — 1 / 2-a 

• m ■ (N"- 1 — 1) = const 

\-n. v ' 



max H(a, m, N) 

OL — 1 , . _ 2-c 

2 - a 
2 < a < 3 

1 < m < (N- 1) 

m G integer (9) 

By solving the mixed integer programming problem, we get the optimal value 
H, the optimal solutions a and m for different average connectivity < k >. 
The results are shown in Fig. 2. 

From Fig. 2, we can see that, in general, with the increase of average 
connectivity < k >, entropy H as well as the minimal connectivity m increases, 
too. In part, the optimal minimal connectivity m keeps constant for a certain 
range of < k >, where scaling exponent a decreases with the increase of < k >. 
As a result, the optimal network becomes much more heterogeneous than the 
one with bigger scaling exponent. 

To prove the entropy of the degree distribution is indeed an effective mea- 
sure of network robustness under node removal, we examine the relationship 
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Fig. 2. The optimal value H, the optimal solutions a and m versus < k > with 
N = 1000. 

between the entropy of the degree distribution and the network threshold 
value p c , which was presented by Cohen et al. with percolation theory [7]. The 
criterion to calculate the percolation critical threshold value p c of a randomly 
connected network is: 



1 ~Pc 



Kq-1 



(10) 



where k = <k ^ is calculated from the original distribution before the random 
failures. With K rs mN^ 1 , kq can be expressed as follows: 



a — 2 . . v 

K » m- (JV«-i -1) 

3 — a 



11) 



Fig. 3. p c and m versus < k > with = 1000. 
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In Fig. 3, p c is calculated from the optimal value a and m which are 
obtained from maximizing H for different average connectivity < k >. The 
results indicate that for some constant m, p c increases with the increase of 
< k >. 

Since both a and N affect network's resilience to random failures, we 
may ask how the network's robustness changes with N. We just examine the 
relationship between p c and a for different N. The results are shown in Fig. 
4. From Fig. 4, we can see that with the increase of a, p c decreases, while the 
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Fig. 4. p c for different N versus a with m = 1. 

threshold value p c becomes bigger with the increase of network scale N, which 
are consistent with the results shown in Fig.l. 

To get an intuitive understanding of the relationship between the two mea- 
sures, we show the results of the threshold value p c and H for different average 
connectivity < k > in Fig. 5. It can be seen that p c and H vary consistently, 
that is, p c increases with H for different average degree < k >. Also, with the 
increase of < k >, both p c and H get improved. 
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Fig. 5. The relationship between p c and H for different average connectivity < k >. 
For all curves, a G (2,3), N G [10 4 ,9 * 10 4 ]. 
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3 Conclusions 



In real-world networks one is often interested in the question of how to 
design a robust network while keeping the cost constant. It is well known that 
scale- free networks are resilient to random failures for their heterogeneous links 
distributions. 

In this paper, different from percolation theory to evaluate the threshold 
value p c , we use the entropy of the degree distribution to study the resilience 
of scale-free networks to random failures. Heterogeneity is a simple but es- 
sential character of scale-free networks and it is in direct relationship with 
the network's resilience to random failures. This character motivates us to use 
the entropy of the degree distribution, which provides an average measure of 
heterogeneity to study the effect of random failures on scale-free networks. 
By optimizing the entropy of the power-law degree distribution, we get the 
optimal value of entropy H, the optimal solutions of a and m for different 

< k >. The results indicate that when the network scale iV is given, the op- 
timal solution m is constant for a certain range of < k >, in which range 
the scaling exponent a decreases. So a higher entropy is obtained. Of course, 
large average connectivity brings high cost. We then examine the relationship 
between threshold value p c and iV for different a. The results are also con- 
sistent with that of the entropy of the degree distribution. We also show the 
relationship of the two measures— p c and H. The results indicate that p c and 
H vary consistently, that is, p c increases with H for a certain average degree 

< k >. Also, both p c and H get improved with the increase of the average 
degree < k >. 

We just try an alternative point of view to analyze the robustness of the 
network from its heterogeneity. The results indicate that the entropy of the 
degree distribution is an effective measure of scale-free network's resilience to 
random failures. Deep discussions on the resilience of scale-free networks to 
targeted attacks will be made in the future. 
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